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- The MAILING DATE of this communication appears on the cover sheet with the correspondence address « 
Period for Reply 

A SHORTENED STATUTORY PERIOD FOR REPLY IS SET TO EXPIRE 3 MONTH(S) FROM 
THE MAILING DATE OF THIS COMMUNICATION. 

- Extensions of time may be available under the provisions of 37 CFR 1.136(a). In no event, however, may a reply be timely filed 
after SIX (6) MONTHS from the mailing date of this communication. 

- If the period for reply specified above is less than thirty (30) days, a reply within the statutory minimum of thirty (30) days will be considered timely. 

- If NO period for reply is specified above, the maximum statutory period will apply and will expire SIX (6) MONTHS from the mailing date of this communication. 

- Failure to reply within the set or extended period for reply will, by statute, cause the application to become ABANDONED (35 U.S.C. § 133). 

- Any reply received by the Office later than three months after the mailing date of this communication, even if timely filed, may reduce any 
earned patent term adjustment. See 37 CFR 1.704(b). 

Status 

1 )□ Responsive to communication(s) filed on . 

2a)D This action is FINAL. 2b)S This action is non-final. 

3) D Since this application is in condition for allowance except for formal matters, prosecution as to the merits is 

closed in accordance with the practice under Ex parte Quay/e, 1935 CD. 11, 453 O.G. 213. 

Disposition of Claims 

4) M Claim(s) 1-26 is/are pending in the application. 

4a) Of the above claim(s) is/are withdrawn from consideration. 

5) D Claim(s) is/are allowed. 

6) S Claim(s) 7-26 is/are rejected. 

7) D Claim(s) is/are objected to. 

8) D Claim(s) are subject to restriction and/or election requirement. 

Application Papers 

9) D The specification is objected to by the Examiner. 

10) 13 The drawing(s) filed on 03 November 2000 is/are: a)D accepted or b)E3 objected to by the Examiner. 

Applicant may not request that any objection to the drawing(s) be held in abeyance. See 37 CFR 1.85(a). 
Replacement drawing sheet(s) including the correction is required if the drawing(s) is objected to. See 37 CFR 1.121(d). 

11) D The oath or declaration is objected to by the Examiner. Note the attached Office Action or form PTO-152. 
Priority under 35 U.S.C. §§119 and 120 

12) D Acknowledgment is made of a claim for foreign priority under 35 U.S.C. § 119(a)-(d) or (f). 

a)D All b)D Some*c)D None of: 

1 .□ Certified copies of the priority documents have been received. 

2. D Certified copies of the priority documents have been received in Application No. . 

3. D Copies of the certified copies of the priority documents have been received in this National Stage 

application from the International Bureau (PCT Rule 17.2(a)). 
* See the attached detailed Office action for a list of the certified copies not received. 

13) D Acknowledgment is made of a claim for domestic priority under 35 U.S.C. § 119(e) (to a provisional application) 

since a specific reference was included in the first sentence of the specification or in an Application Data Sheet. 
37 CFR 1.78. 

a) □ The translation of the foreign language provisional application has been received. 

14) D Acknowledgment is made of a claim for domestic priority under 35 U.S.C. §§ 120 and/or 121 since a specific 

reference was included in the first sentence of the specification or in an Application Data Sheet. 37 CFR 1.78. 
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1 ) Notice of References Cited (PTO-892) 4) □ Interview Summary (PTO-413) Paper No(s). 

2) CD Notice of Draftsperson's Patent Drawing Review (PTO-948) 5) O Notice of Informal Patent Application (PTO-1 52) 

3) |3 Information Disclosure Statement(s) (PTO-1 449) Paper No(s) 2 . 6) □ Other: 
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DETAILED ACTION 
Drawings 

1 . The drawings are objected to because the graphs shown in Figures 1 and 2 are hard to 
read because the axis labels are too small. Larger graphs or labels are needed. A proposed 
drawing correction or corrected drawings are required in reply to the Office action to avoid 
abandonment of the application. The objection to the drawings will not be held in abeyance. 

Claim Objections 

2. Claims 1 and 13 are objected to because of the following informalities: step (e) in both 
claims states, "computing a measure of similarly between the input image and each rendered 
image" (emphasis added). The examiner believes that applicant intended to use the word 
"similarity" in place of the word "similarly " Appropriate correction is required. 

Claim Rejections - 35 USC § 103 

3. The following is a quotation of 35 U.S. C. 103(a) which forms the basis for all 

obviousness rejections set forth in this Office action: 

(a) A patent may not be obtained though the invention is not identically disclosed or described as set forth in 
section 102 of this title, if the differences between the subject matter sought to be patented and the prior art are 
such that the subject matter as a whole would have been obvious at the time the invention was made to a person 
having ordinary skill in the art to which said subject matter pertains. Patentability shall not be negatived by the 
manner in which the invention was made. 

4. Claims 1-8, 1 1-20, and 23 are rejected under 35 U.S.C. 103(a) as being unpatentable over 
Chang et al. (USPN 6,137,896, newly cited), hereafter Chang, in view of Belhumeur et al. 
("Eigenfaces vs. Fisherfaces: Recognition using class specific linear projection," IEEE 
Transactions on Pattern Analysis and Machine Intelligence, newly cited), hereafter Belhumeur. 

5. Regarding claims 1 and 13, Chang discloses a method for choosing an image from a 
plurality of three-dimensional models (column 2, line 39-55) which is most similar to an input 
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image (column 4, line 46-57); the method comprising the steps of: (a) providing a database of the 
plurality of three-dimensional (rendered images determined from a range image) models (column 
6, line 46-51; column 10, line 4-9); (b) providing an input image (column 7, line 49-51); (c) 
positioning (orienting) each three-dimensional model (reference range image) relative to the 
input image (column 9, line 10-14); (d) for each three-dimensional model, determining a 
rendered image that is most similar to the input image (column 9, line 21-35); (e) computing a 
measure of similarly (correlation value) between the input image and each rendered image 
(column 4, line 46-55); and (f) selecting the three-dimensional model (reference images) 
corresponding to the rendered image whose measure of similarity is most similar to the input 
image (column 9, line 26-35). 

6. Chang discloses that correlation results between images are substantially illumination 
independent (column 2, line 53-54; column 5, line 41-61), but does not disclose a step of 
computing a linear subspace that describes an approximation to the set of all possible rendered 
images under all possible lighting conditions, and finding the point on the linear subspace that is 
closest to the input image or projecting the set of images generated by positive lights onto the 
linear subspace. Belhumeur discloses a comparison of Eigenfaces and Fisherfaces, two 
recognition systems using class specific linear projections, comprising: (i) computing a linear 
subspace that describes an approximation to the set of all possible rendered images that each 
three-dimensional model can produce under all possible lighting conditions where each point in 
the linear subspace represents a possible image (page 711, left column, line 15-right column, line 
6); and (ii) finding the point on the linear subspace that is closest (shortest distance) to the input 
image (page 713, right column, line 9-22) or (ii) finding a rendered image in a subset of the 
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linear subspace obtained by projecting the set of images (page 1, left column, line 26-right 
column, line 6) that are generated by positive lights (Lambertian surfaces without shadowing) 
onto the linear subspace (page 713, left column, line 43-60). It would have been obvious to one 
of ordinary skill in the art at the time the invention was made to compute a linear subspace 
describing an approximation to the set of all possible rendered images under all possible lighting 
conditions, find an image in a subset of the linear subspace that is closes to the input image as 
taught by Belhumeur in order to achieve error free classification under any lighting conditions 
provided the surfaces obey the Lambertian reflectance model (page 713, right column, line 23- 
26). See also page 713, bottom of right column and page 714 regarding the Fisherface method. 

7. Regarding claims 2 and 14, Belhumeur discloses that step (a) comprises building each 
three-dimensional model from a series of images (three) taken under predetermined lighting 
conditions (page 713, left column, line 54-right column, line 13). 

8. Regarding claims 3 and 15, Chang discloses that step (a) comprises assigning a location 
to each point on the surface of each three-dimensional model (range image) and at least one 
corresponding identifier (surface normal vector), which identifies the fraction of light that is 
reflected at each point (column 5, line 1-61). 

9. Regarding claims 4 and 16, Belhumeur discloses that the at least one corresponding 
identifier (intensity) comprises albedos (page 713, left column, line 43-55), but does not disclose 
that the identifier comprises three albedos, one for each of how much red, blue, and green light is 
reflected. The examiner takes Official Notice that splitting light into three separate components 
corresponding to red, green, and blue colors of light is well known in image processing and other 
arts. It would have been obvious to one of ordinary skill in the art at the time the invention was 
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made to determine an identifier comprising three albedos, one for each of how much red, blue, 
and green light is reflected in order to treat each component separately for filtering, recognition, 
processing, etc. 

10. Regarding claims 5 and 17, Chang discloses that step (b) comprises providing a two- 
dimensional input image (column 9, line 36-38). 

1 1 . Regarding claims 6 and 1 8, Belhumeur discloses that step (c) comprises aligning 
predetermined points on the three-dimensional model and the input image (page 712, left 
column, line 37-40). 

12. Regarding claims 7 and 19, Chang discloses that the transformed input image is filtered 
using a filter derived from and associated with a three-dimensional image to produce a 
correlation result, and that the process is repeated until all filters (three-dimensional models) are 
exhausted or a sharp correlation peak is detected (column 9, line 21-35), but does not disclose 
that step (d) is repeated for each of a red, green, and blue color component for each three- 
dimensional model. See above discussion of claims 4 and 16 with regard to red, green, and blue 
color components. 

13. Regarding claims 8 and 20, Belhumeur discloses that step (d)(i) comprises computing 
polynomials from descriptions of the direction of the surface normal at each point (n(p)) and 
from the at least one corresponding identifier (albedo a(p); page 713, left column, line 43-55). 

14. Regarding claims 1 1 and 23, Chang discloses that step (e) comprises determining the 
magnitude of the difference (correlation) between the input image and each rendered image 
(column 6, line 34-43). 
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15. Regarding claim 12, Belhumeur discloses that step (d)(ii) comprises computing the point 
in the linear subspace that is closes to the input image using a linear projection (page 713, right 
column, line 13-16). 

16. Claims 9, 10, 21, 22, and 24-26 are rejected under 35 U.S.C. 103(a) as being unpatentable 
over Chang in view of Belhumeur as applied to claims 1 and 13 above, and further in view of 
Gotsman et al. (USPN 6,501,857, newly cited), hereafter Gotsman. 

17. Regarding claims 9, 10, 21, and 22, neither Chang nor Belhumeur discloses that the linear 
subspace is either four- or nine-dimensional. Gotsman discloses a method and system for 
detecting and classifying objects in an image wherein the system processes directory images to 
obtain eigenvectors and eigenvalues, and selects a set of basis vectors (preferably 3-9 basis 
vectors) formed by linear combinations of theses eigenvectors to be applied against a target 
image (abstract; column 10, line 32-36). It would have been obvious to one of ordinary skill in 
the art at the time the invention was made to utilize a linear subspace (defined by a set of basis 
vectors) having dimensions between three and nine as taught by Gotsman in order to satisfy the 
requirements of the particular system in order to avoid false matches because of too few basis 
vectors, and false rejections because of too many basis vectors (column 10, line 36-41). 

18. Regarding claim 24, Chang discloses rendering an image of each three-dimensional 
model using non-negative lighting (column 6, line 44-51), but does not disclose that it is done by 
solving a sixth order polynomial. The examiner considers this limitation to be a design choice 
that depends on the particular application of the method. It would have been obvious to one of 
ordinary skill in the art at the time the invention was made to render an image of each three- 
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dimensional model using non-negative lighting by solving a sixth order polynomial in order to 
obtain the desired two-dimensional image view from the three-dimensional model for 
comparison and other processing. 

19. Regarding claim 25, Belhumeur discloses that step (d)(ii) comprises finding the rendered 
image that is a convex (page 713, right column, line 42-50) combination of images generated 
with light coming from a single direction (page 713, left column, line 55-right column, line 8), 
projected onto the nine-dimensional space (see above discussion of claims 10 and 22 with 
respect to Gotsman). 

20. Regarding claim 26, Gotsman discloses that the rendered image (detected match) is found 
using a non-negative least squares (differences below threshold) algorithm (column 13, line 63- 
column 14, line 9). 

Conclusion 

2 1 . The prior art made of record and not relied upon is considered pertinent to applicant's 
disclosure. 

• USPN 5,710,833 to Moghaddam et al. discloses detection, recognition, and coding of 
complex objects using probabilistic eigenspace analysis. 

• USPN 5,724,447 to Fukushima discloses an optical transform system for three- 
dimensional object recognition. 

• USPN 6,009,437 to Jacobs discloses linear fitting with missing data: applications to 
structure-from-motion and to characterizing intensity images. 
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• USPN 6,292,575 to Bortolussi et al. discloses a real-time facial recognition and 
verification system including projections in image sub-space using principal 
component analysis and compensation for contrast and brightness differences. 

• USPN 6,466,685 to Fukui et al. discloses a pattern recognition apparatus and method 
including adopting a subspace independent of change in illumination. 

• USPN 6,621,929 to Lai et al. discloses a method for matching images using spatially- 
varying illumination change models. 

• WO 00/33240 to Taylor et al. discloses face sub-space determination making initial 
estimates of lighting, pose, identity, and expression using principle component 
analysis. 

• Nayar et al. discloses "Dimensionality of illumination in appearance matching" as a 
robust and efficient approach to 3D object recognition and pose estimation wherein 
each object is represented in a low-dimensional subspace. 

• Erturk et al. discloses "3D model representation using spherical harmonics" wherein 
the amount of data is significantly reduced. 

• Adini et al. discloses "Face recognition: the problem of compensating for changes in 
illumination direction" to recognize a face fro a novel image despite variations 
between images of the same face. 

• Hager et al. discloses "Efficient region tracking with parametric models of geometry 
and illumination." 

• Jacobs et al. discloses "Comparing images under variable illumination " 

• Chen et al. discloses "In search of illumination invariants." 
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Any inquiry concerning this communication or earlier communications from the 
examiner should be directed to Ryan J Hesseltine whose telephone number is 703-306-4069. 
The examiner can normally be reached on Monday - Friday, 8:30 AM - 5 PM. 

If attempts to reach the examiner by telephone are unsuccessful, the examiner's 
supervisor, Amelia Au can be reached on 703-308-6604. The fax phone number for the 
organization where this application or proceeding is assigned is 703-872-9306. 

Information regarding the status of an application may be obtained from the Patent 
Application Information Retrieval (PAIR) system. Status information for published applications 
may be obtained from either Private PAIR or Public PAIR. Status information for unpublished 
applications is available through Private PAIR only. For more information about the PAIR 
system, see http://pair-direct.uspto.gov. Should you have questions on access to the Private PAIR 
system, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). 
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